Performance Measurement Intrusion and Perturbation Analysis

Software performance instrumentation perturbs the state of the measured system. The primary source of this perturbation is the execution of additional instructions. However, ancillary perturbations include disabled compiler optimizations and memory conflicts. Collectively, these perturbations can increase the measured system’s execution time, change memory reference patterns, reorder events, and even cause register interlock stalls. The perturbation magnitude depends on the intended performance measurements and the granularity of the instrumentation. In this paper, we study the instrumentation perturbations of software event tracing on the Alliant FX/SO vector multiprocessor in sequential, vector, concurrent, and vector-concurrent modes. Based on experimental data, we derive a perturbation model that can approximate true performance from instrumented execution. We analyze the effects of instrumentation coverage (i.e., the ratio of instrumented to executed statements), source level instrumentation, and hardware interactions. Our results show that perturbations in execution times for complete trace instrumentation can exceed three orders of magnitude. With appropriate models of performance perturbation, these perturbations in execution time can be reduced to less than 20% while retaining the additional information from detailed traces. In general, we conclude that it is possible to characterize perturbations through simple models. This permits more detailed, accurate instrumentation than traditionally believed possible.